An Adaptive Multiple Feature Subset Method for Feature Ranking and Feature Selection

In this paper, we propose a new feature evaluation method that forms the basis for feature ranking and feature selection. The method starts by generating a number of feature subsets in a random fashion and evaluates features based on the derived subsets. It then proceeds in a number of stages. In each stage, it inputs the features whose ranks in the previous stage were above the median rank and re-evaluates those features in the same fashion as it did in the first stage. When the number of features is high, the method has a computational advantage over recursive feature elimination (RFE), a state-of-art method that ranks features by identifying the least valuable feature in each stage. It also achieves better results than RFE in terms of classification accuracy and some other measures introduced in this paper, especially when the size of the training data is small or the number of irrelevant features is large.